5 Statistical Data Editing

Statistical Data Editing (SDE) is the process of checking data for errors and correcting them. Winkler (1999) defined it as the set of methods used to edit (i.e., clean up) and impute (fill in) missing or contradictory data. The result of SDE is data that can be used for analytic purposes. Editing literature goes back to the 1960s with the contributions of Nordbotten (1965), Pritzker, et al. (1965), and Freund and Hartley (1967). A first mathematical formalization of the editing process was given by Naus, et al. (1972), who introduced a probabilistic criterion for the identification of records (or part of them) that failed the editing process. A solid methodology for generalized editing and imputation systems was developed by Fellegi and Holt (1976). The great break in rationalizing the process came as a direct consequence of the PC evolution in the 1980s, when editing started to be performed online on personal computers, even during the interview and by the respondent in CASI models of data collection (Bethlehem et al., 1989). Nowadays, SDE is a research topic in both academia and statistical agencies. The European Economic Commission organizes a yearly workshop on the subject that reveals an increasing interest in both scientific and managerial aspects of SDE.